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ABSTRACT 


Sixteen subjects attempted to replicate the intensity of a criterion 
Stimulus after one and two cross-modality matches. Matches involved 
visual or auditory criterion stimuli matched to kinaesthetic, visual 
and auditory output. It was found that within subjects, the number 
of matches made in each trial had no effect on the memory of the 


criterion stimulus. 
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INTRODUCTION 


It has been asserted (Pribram, 1971) that a comparison process 
is involved in associations between sensory modalities. Current 
sensory excitation is screened against some representative record 
of past experience. 

Recently there have been detailed associational mechanisms 
suggested. Konorski (1967) proposed a system of directional connec- 
tions between units of specific storage areas. Luria (1973) hypothe- 
sized the existence of higher associational areas formed by the over- 
lapping of cortical ends of various analyzers. Evidence of specific 
dimensions for comparisons between specific modalities has been found 
by Auerbach and Sperling (1974), Connolly and Jones (1970) and is 
suggested by Signal Detection Theory (Coombs et al, 1970). A more 
extensive review of literature can be found in Appendix II. 

Pribram (1971) suggests that association between modalities 
occurs through hits of the primary projection areas in the brain that 
are sensitive to excitation in a modality different (cross-modality) 
than the major sensory mode served by that system. The following 
experiment uses this concept of multi-modality matching to compare 
the accuracy in matching over two and three different modalities, 
namely; visual, auditory, and kinaesthetic. A statement of the pro- 


blem is found in Appendix I. 
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EXPERIMENTAL DESIGN 


Two conditions were presented in a factorial design with two 
modalities. The modalities consisted of criterion stimuli presented 
in visual (V) and auditory (A) modes. Visual stimuli were intensities 
of light at)5.0,) S875, h2 7a 7loezoaand.20;0GWatts. + Auditory stimuli 
were loudnesses sof sound at 59.0:.-65..52./225,. /4.3-and /75.8-dB. 

The first condition consisted of a criterion stimulus presented 
for five seconds. “The subject (S) was: required to judge the intensity 
of the stimulus and to produce an extensor arm movement (K) of a 
length corresponding to that intensity. He was then asked to use his 
appropriate control knob to try to reproduce the intensity of the 
original criterion stimulus. Thus, the criterion was presented, S 
gave a kinaesthetic response (K) and then a response in the original 
modality. For example, a visual criterion was followed by a kinaesthe- 
tic response and then a visual response. This is shown as V1?K~V2, 
where 1 indicates a presented stimulus and 2 a response (K is always 
a response). 

Condition Two used the same criterion stimuli, but one additional 
response was required before the K response and the attempt to repro- 
duce the original. For example, a visual criterion was followed by an 
auditory response, a kinaesthetic response and then the final visual 
response. This is shown as NI-P h2—>K-7 V2. Table 1 shows the four 


combinations used. 
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Table 1 
Experimental Design for the Cross-—Modality 


Matches Given to all Subjects 


_ Modality 
Visual Auditory 
Condition One Vi->K-V2 Al—>K-» A2 
Condition Two VI-rA2->K-> V2 Al->V2—>KvA2 


Ten trials in random order (five criterion levels given twice 
each) were given on each combination. The order of the combinations 
was varied between subjects, but Condition One always preceded Condi- 


tion, 1WO. 
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METHOD 


Independent Variables 

Visual intensity (V) and auditory loudness (A) were matched to 
the kinaesthetic sense of length of arm movement (K) using a cross- 
modality matching (CMM) oar See Appendix III for specific defini- 


tions of terms, stimulus variables and résponse. variables. 


Subjects 


Sixteen summer session students and secretaries were paid $2.00 


éach to be subjects (S) in the experiment. 


Apparatus 


The apparatus consisted of devices for measuring K, V and A. 
Figures 1 and 2 illustrate the apparatus used. A wooden bar cali- 
brated in centimetres was equipped with two metal clamps as stops 
and a freely moveable cursor with a handle. This was used to measure 
the length of the S's extensor arm movements (K) directly in centi- 
metres. S could reach the cursor easily with his right hand, but could 
not see it as his right arm and hand were covered by a cloth fastened 
around his neck. The experimenter (E) had a clear, illuminated view 
of the wooden bar. 

The device for measuring V was a 25 Watt light bulb connected to 
a variable transformer which varied the power to the light bulb over 
a range of 0 to 25 Watts. S viewed the light bulb through a small 
hole at his eye level. The variable transformer was equipped with a 


dial which E could read and manipulate. 
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FIGURE 1 


APPARATUS: SUBJECT'S SIDE 
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FIGURE 1 


APPARATUS: OPERATOR'S SIDE 
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With his left hand, S could reach a knob directly connected to the 
dial of the variable transformer with which he could vary the inten- 
sity of the light. 

A potentiometer calibrated in decibels was connected in series 
with a speaker, tone generator and amplifier to regulate the loud- 
ness of a 1000 Hz tone. The potentiometer, similar to the variable 
transformer, had a dial which could be controlled by S or E to vary 
the: doudness ofthes tone (A).<* EB) controlled cireuit switches for both 
the V and A devices and had manual access to the K measurement. See 
Appendix IV for actual experimental procedure and instructions to sub- 


jects. 
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RESULTS 


Absolute (AE) and algebraic error (CE) scores were computed for 
each trial by comparing the presented intensities with final responses 
in the same Sa cibeaen (See Appendix V.) A four-way analysis of vari- 
ance was performed on AE scores. There was no significant difference 
found between conditions. The similarity between conditions is illus- 
trated by the graphs in Figures 3 and 4. This suggests that the cri- 
terion level of sensitivity stays constant with multi-modality matching. 
Modality differences were not comparable. 

Variable error (VE) scores (standard deviation from algebraic 
mean) gave similar results for the two conditions. This analysis 
Showed. Ssaeniticant subject differences, -F (15935). = 19.7, p< 0.01. 
This supports the work of Rule and Markley (1971) who found subject 
differences and concluded that they were due to idiosyncratic use 
of the Bee eden eae cee No interactions were found significant 
except the subject by repetition interaction. This indicates that 


over trials, subjects became more precise. 
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FIGURE 3 
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DISCUSSION 


oruite reported here have indicated that a beta level (criterion 
level of sensitivity) is set up and stays constant with multi-modality 
matching. If the match was made on the basis of the preceding feed- 
back, the regression noted by Stevens (1969) would alter the beta level 
for the following match. The consistency of the matching in this exper- 
iment suggests that a central beta level is established common to all 
modalities, denying the connectionist approach to association. 

Pribram's (1971) holographic model for memory storage and asso- 
ciation is compatible with these results. His theory would have the 
beta level stored in units of the primary projection cortex that are 
sensitive to inter-modal associations. The work of Auerbach and Sper- 
ling (1974), Connolly and Jones (1970) and Signal Detection Theory 
(Coombs et al, 1970) also propose a central decision axis or storage 
system of some kind. No specific structures are proposed by these 
authors. 

Luria (1973) suggests a central mechanism at a higher level rather 
than within primary systems. This experiment makes no distinction as 
to level of association, but Luria's model would involve complex con- 
nections that have not been substantiated. 

The efficiency of the human ee Supports the extension of the 
associational information reported here for visual auditory and kinaes- 
‘thetic modes to include all sensory modalities in a model of associa- 


tion using a central decision axis. 
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APPENDIX I 


THE PROBLEM 


Stevens (1969) has noted that a regression effect or centering 
tendency occurs in CMM experiments. The subject tends to shorten 
the range of the stimulus variable he controls. When multiple matches 
are made, this regression error may be compounded if the match is made 
on the basis of feedback (input) from the immediately preceding match. 
This would imply a connectionist association mechanism. If, however, 
the match is made against a central sensitivity level common to all 
modalities, the regression error should be minimal and not compounding. 

The problem may be clearly stated as the following questions: 
Does the criterion level of sensitivity va level) stay constant with 
multi-modality matching? Does the error due to regression in the 
manipulated modality alter the 3 level for further matches? What 


are the associational implications of the results? 
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APPENDIX II 


REVIEW OF LITERATURE 


Sensory Perception 

The properties of the sensory modalities and their interactions 
are important for efficient human performance. Hartshorne (1934) 
stressed the idea of separation between the modes of sensation. This 
idea has been developed into a generally accepted concept of division 
of the brain into modality specific areas. Information specific to 
each modality is thought to be stored in a corresponding area of the 
brain. Visual information, for example, is centered in the occipital 
lobe of the cerebrum, while audition is located in the temporal lobe. 
Motor areas are not so highly localized, although they are centered 
in the posterior section of the frontal lobe with the visual motor 
area in the parietal lobe (Grossman, 1973). 

Each modality has peripherally located arene that are sti- 
mulated by incoming coma on This stimulation is coded into an 
array of nerve impulses which reach the synaptic networks of the pri- 
mary projection area. This pattern of impulses is coded into the 
brain's continuous pattern of slow potentials to produce an internal 


representation or coded image of the stimulus (Pribram, 1971). The 


patterns of encoded stimuli are decoded into patterns of nerve impulses, 


transmitting information to the appropriate storage centres (associa- 
tion cortex). 
Posner's (1969) concept of the coded image is that of a spatial 


array that can be searched. Pribram (1971) takes this idea further. 
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He suggests that since the movement of the slow potentials is some- 
what describable as wave forms, the coded images may be similar to 
holographic images. Holograms are described by the same convolution 
integrals used for analysis of wave interference patterns. Essen- 
tially, the neural equivalent is spatial interactions among phase 
relationships of neighbouring junctional patterns. A physical process 
_ based on interference effects (resulting from the lateral inhibition 
occurring in neural events) displays many of the attributes of the 
neural process of perception. This holographic type of transforma- 
tion, combined with feature filters, describes a mechanism of crea- 
ting a neural image of stimuli. These images can be decoded for storage 
and reconstructed at will. Further holographic features are that many 
different patterns can be simultaneously stored and individually re- 
trieved from the same storage surface, and that a whole, three dimen- 
sional image may be reconstructed from any part of the storage area 
(Pribram, 1971, Ch. 8). Such a process readily describes a mechanism 
of image formation and the capacities of recognition memory. 
Konorski's (1967) theory of gnostic units as units of recognition 
is based on the concept of neural activity as all or none firing of 
individual neurons. He stresses the importance of synaptic modifica- 
tion through repeated stimulation to establish learned pathways to re- 
cognition units as well as directional associative links between units. 
This idea is compatible with the slow potentials and holographic 
coding hypothesis. The patterns of stimulation resulting from decoded 
images require synaptic transmission to points of storage that can be 


stimulated to recode the stored interference patterns. 
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Pribram (1971) suggests that although research on spacial inter- 
actions in junctional patterns has centered on the visual system, the 
slaivoe and kinaesthetic systems may provide similar models. 

Posner (1973), names three qualitatively different codes with 
which this study in concerned; iconic codes to represent visual 
images, echoic codes for auditory images, and enactive codes to repre- 
sent motor programs. Differences between these codes appear when 
attentional factors are considered. All three decay rapidly with time, 
although iconic and echoic retention can be aided by the availability 
of attention (rehearsal). However, attentional factors appear to have 


a much reduced effect on the storage and performance of enactive codes. 


Association 

In order for the organism to act as a functional unit, there must 
be extensive associations established between the sensory systems. 
Pristotler(e ted. it Takatura.. 19/) edn thes fourth century B.C. ,..de- 
fined a first sense, "“sensus communis", as a central organ common to 
all special sensations. Special sensations were termed as those which 
act in terms of special organs such as the eye, ear, or nose. The 
sensus communis was assigned responsibility for the comparison, des- 
crimination, and the unity of special sensations. 

Co-ordinated, unified activity of the brain appears to require ex- 
tensive associational connections between the sensory modalities. Codes 
of different types are assumed to be of different formats. Consequently, 
if matching is to occur between modalities, internal transformation of 


information from the storage code to the appropriate code for 
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comparison must occur. Swanson et al (1972) termed that operation 
"recoding." 

Although the actual mechanisms of association are not understood, 
there is most likely a comparison process involved. Pribram (1971) 
asserts that: 

At any moment, current sensory excitation 
is screened by some representative record 
of prior experience; this comparison - the 
match between current excitation and re- 
presentative record - guides attention and 
action. (py, 49) 

Recently there have been detailed associational mechanisms sug- 
gested. Konorski (1967) proposed a system of directional connections 
between units of specific storage areas. Thede connections are neural 
pathways that could be established by synaptic modifications as exper- 
ience and importance to the organism dictate. 

An alternative to this connectionist approach hypothesizes the 
existence of a higher associational area which operates by convergence 
of information from the modality specific storage areas. Luria (1973) 
proposes a specifically human structure of overlapping of the cortical 
ends of various analyzers to form areas concerned with the integration 
of excitation arriving from eee different systems. 

Auerbach and Sperling (1974) ea evidence for the existence 
of a single direction dimension common to auditory and visual direc- 
tion for purposes of comparison. Connolly and Jones (1970) propose 
the existence of a long-term integrated storage system which contains 
some internal representation of the relationship between visual and 


kinaesthetic information to facilitate translation between these 


modalities. Multi-dimensional stimulus inputs are presumed to be 
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mapped into a Sieh dimenenenel decision axis in Signal Detection Theory 
(Coobs et al, 1970). These contributions may imply a higher associa- 
tive area or merely affirm commonalities between modalities at te 
mary levels. 

Pribram (1971) doubts the existence of associational connections 
between areas, or that there is a solely human associational structure. 
He suggests that association occurs through units of the primary pro- 
jection areas that are sensitive to eeeiuaedon in a modality different 
than the major sensory mode served by that system. He notes that 
lesions in the primary projection cortex disturb inter-modal associa- 
tions more than lesions in the so-called association cortex. One of 
the attributes of the holographic hypothesis, a mechanism of the pri- 
Mary projection areas, is facility in associative recall. Many connec- 
tions and fibre tracts exist between areas of the brain and between 
the left and right hemispheres. These connections serve to distribute 
coded representations throughout the brain and are suggested to have 


inhibitory rather than associational functions (Pribram, 1971). 


The Measurement of Sensation 
Perception of a stimulus involves transformations of the original 
energy received, so that what exists may not be the same as what is 
perceived. The problem then, is how to accurately measure sensation. 
Fechner (1860) first measured sensation in terms of successive 
increments of just noticeable differences (jnd's). He assumed all 
jnds to be subjectively equal, and to. represent constant differences 


in sensation. His work resulted in the relation 
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where al is the magnitude of psychological sensation, D is the sti- 
mulus magnitude and k is a constant dependent on units of #. Stevens' 
psychophysical law is based on the concept of a jnd as a value which 
is a constant proportion of the original stimulus, so that jnds are 
constant ratios of sensation (Miller, 1959). However, Krantz (1972) 
would argue that sensations are internal mediators, not numbers, so 
; the term "ratio" is undefined in this case. Stein (1975) disputes 
the use of the term "law" in relation to the psychophysical law. The 
only scientifically lawful relation between stimulus and sensation is 
the hyperbolic relation which exists between receptor potential and 
conductance change after encoding (Stein, 1975). 
The much debated psychophysical law states that: 

Every sensory continuum exhibits the same 

invariance: equal stimulus ratios produce 

equal sensation ratios. (Stevens, 1966, 

p+ 5) 
Stevens has used this law with methods of magnitude estimation (num- 
bers are assigned in proportion ve sensory magnitudes) to find a rela- 
tion between sensation and stimulus: 

The sensation magnitude / grows as a power 

function of the stimulus magnitude @ . In 

terms of a formula, 

w= kp”. (Stevens, 1959, p. 115) 
The exponent, n, of the power function is roughly a constant expres- 
sion of the rate of growth of sensation magnitude, specific to each 
‘sensory continuum. 
There are some problems associated with this function. The size 


of the exponent may vary with stimulus range used, in that a smaller 


range results in a larger exponent. Engeland and Dawson (1974) found 
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persisting individual differences between subjects. Procedural dif- 
ferences can alter the exponent, methods of magnitude production 
giving a larger exponent than magnitude estimation (Marks, 1974). 
Magnitude estimation requires assigning numbers to given stimulus 
intensities, while in magnitude production, the experimenter gives 
numbers to be matched by adjusting the stimulus intensity. 

Krantz (1972) prefers the method of relation theory to these mag- 
nitude estimations. In relation aieapee any stimulus is judged in a 
context of previous an concurrently present stimuli, and judgments 
are mediated by perceived relations pertaining to pairs of stimuli. 

He points out that this latter method more closely approaches what he 
actually does when comparing two stimuli. This relation theory is the 


basis for cross-modality matching. 


Cross—Modality Matching 


Cross-modality matching (CMM) experiments require that subjects 
match the magnitude of sensations produced by stimuli of one modality 
to sensations produced by stimuli of another (Auerbach, 1973). CMMs 
are based on the discriminability of stimuli rather than on the magni- 
tude of sensations they produce, so the questionable validity of mea- 
suring magnitude of sensations is circumvented. 

Signal Detection Theory has been established with a beta (2) value 
as a level of sensitivity common to all stimulus inputs. Signal detec- 
tion operates on the basis of a known likelihood ratio which expresses 
the probability distribution of the occurrence of the signal. A beta 


level is determined using factors of prior odds and values of responses 
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as a cut-off level or threshold for comparison oeges the likelihood 
ratio (Coombs et al, 1970). In the present study, the probability 
of mocarrante is always one and the value of a correct response is 
always of prime importance. As well, there is no problem distin- 
guishing signal from noise, as the given signal is always well above 
threshold. A beta value established in this CMM case would provide 
a decision axis for discrimination of an equal sensation in another 
: modality. Thus, this beta value shall be considered the basis for 


comparison in cross~modality matches. 
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APPENDIX III 


Definitions of Terms 

CMM: Cross-modality matching; this indicates the matching of re- 
ceived stimulus magnitude of a sensory continuum in one modality 
with output of sensory magnitude on a second sensory continuum 
in a second modality. 

Beta (8): a level of sensitivity determined by perception of an input 
stimulus; a decision axis (same or different) for matching of 


further variables. 


|n 


subject 


E: experimenter 


Stimulus Variables (Independent Variables) 
Al: auditory modality; input given as loudness of tone. 


Vl: visual modality; input given as brightness of light. 


Response Variables (Dependent Variables) 
K: kinaesthetic modality; operationally measured as length of arm 
movement (extensor) originated by S to match V1 or Al. 
V2: visual modality; operationally measured as the brightness of a 
light, manipulated by S to match V1 or Al. 
A2: auditory modality; operationally measured as loudness of sound, 


manipulated by S to match V1 or Al. 
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APPENDIX IV 
EXPERIMENTAL PROCEDURE 

Each subject was placed carefully in a chair, so that he was 
lined up with the light source and the speaker and was within reach 
of the control knobs and K cursor. He was briefly instructed on the 
operation of the control knobs and was allowed to see and try out the 
cursor. Then a black cloth was fastened around his neck leaving his 
left hand free to reach the control knobs and his right arm and hand 
hidden to manipulate the cursor unseen by himself. The lights were 
then turned off and instructions given as S became dark adapted. 

The instructions to subjects were as follows: 

"There are three factors which you can control: brightness 

of light, loudness of tone and length of arm movement. 
Movement length is just the natural extension of the arm. 
Trystheiuange offbrightnessiof lightyy (S tried it). 
Try the range of loudness of tone. (S tried it). I will 
give you an intensity of light or a loudness of sound for 
five seconds. Concentrate on it. After the signal finishes, 
you will be asked to respond on the basis of that signal. 
There are four response orders. Before the experiment 
begins, we will have a few practise trials on each order. 

Here were given two practise trials on each of the first two 
configurations to be given (Condition 1) and one on each of the last 
two (Condition 2). Practises were given in the same order as the 
actual testing. Each subject received the same intensities for prac- 
tise Clight=at 40 and 70 units, sound at.60.4 and 73.5 dB). Any ques- 


tions as to procedure were answered and subjects were requested to 


respond as accurately as possible on all attempts. 
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APPENDIX V 


Table 2 


Mean Error Scores 


for Modalities and Conditions 


AE CE VE AV" "* 
Visual 6.44 LOL 500 7.93 
Auditory ace L 2955 2250 2283 
Condition 1 4.88 Dea 2 4.19 SyeeKs 
Condition -2 4.98 1358 3297 Bz 


*AF = Absolute error 

CE = Constant (Algaebraic error) 
VE = Variable error 

AV = Average variation 
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APPENDIX VI 


IMPLICATIONS FOR PHYSICAL EDUCATION 


Every physical educator makes use of the cross-modality 
matching phenomenon when teaching motor skills. A verbal expia- 
nation accompanied by a demonstration gives the student visual 
and ane input to match with kinaesthetic output as he tries 
ie perform the movement. Whether this match takes place against 
a central stored image or peripherally between modalities will 
affect the learning and teaching processes. 

This thesis supports the theory that a central image is 
formed for comparison between all sensory modalities. This allows 
the teacher to draw on any appropriate comparison that would suggest 
the desired movement and helpcreate an accurate image. For fee tao 
a dancer may find a quality of movement suggested by a colour. 
Similarly, the colour red may suggest power to a volleyball player 
attempting a eee There are few limits to the associations possible, 
and the inventive teacher will use cross-modality matching where needed 
to strengthen or correct a student's image of the movement (motor 
programme). 

At times the student will have an incorrect image that must be 
altered or replaced. These bad habits are usually well learned by the 
kinaesthetic system, and the correction process can be long and arduous. 

Awareness of the correct image is essential as is the student's 
awareness of how his own performance differs from that image. Visual 


feedback using mirrors or, better yet, videotapes will assist the student 
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in feeling the image of his movement. Once the student is aware of 
how he is moving by using as much sensory input as possible to feel 


the movement, learning will be facilitated. 
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APPENDIX VII 


RAW DATA 


Data is presented as written down at the time of experiment. 
For analysis, some transformations must be made. Kinaesthetic 
measurements must be subtracted from 79 to give the actual length 
of arm extension in centimetres (K). Visual measurements (V1, V2) 
were recorded as units on the dial cf the variable transformer, 
where 1 unit = 0.25 Watts. Auditory (Al, A2) measurements were re- 
corded as degrees of a circle describing the position of the pointer. 
The formula to convert this position to decibels is as follows: 
y = 0.03801 x + 0.00013 ne +54. 30163 


where x is the reading in degrees (°) and y is the equivalent in dB. 
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SUBJECT 1 
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